1. Field of the Invention
The present invention relates to light source estimation.
2. Description of the Related Art
There is a case where image processing by image processing software etc. is performed in order to give the effect intended by a person who has captured an image to the captured image. For example, mention is made of image processing to change the kind and direction of a light source of the captured image, to synthesize a virtual object, etc. Such image processing has required a skilled person to spend much time to create a processed image without unnaturalness, however, it has been made possible to automatically perform image processing based on information, such as a subject shape, texture, a light source in an image capturing scene, etc. Further, various kinds of techniques to estimate such information from a captured image have been proposed.
For example, there is a technique to estimate a light source by arranging a special apparatus, such as a two-dimensional marker provided with a specular sphere in an image capturing scene. In this case, it is possible to estimate the light source by performing calculation processing based on the position of a camera and the captured image of the specular sphere from which the light source is reflected. However, it is necessary to provide the special apparatus as described above and to capture an image after arranging the special apparatus in an image capturing scene, and therefore, there has been such a problem that much effort and time are required. In such circumstances, a method for estimating a light source by acquiring an entire periphery image by a user provided with a head mount display to which a video camera is attached moving about is proposed (Japanese Patent Laid-Open No. 2008-33531).
There is also a technique to compare CG image data obtained by adjusting light source parameters and rendering a subject shape under various kinds of light source on condition that the shape and texture of the subject be already known, and actually photographed image data, and to estimate a light source by which a difference therebetween is the smallest as a light source in the real scene (Takahiro Okabe, “Spherical harmonics vs. Haar wavelets: basis for recovering illumination from cast shadows” Proc. IEEE Conf. Computer Vision and Pattern Analysis (CVPR 04), pp. I-50-57, 2004, hereinafter referred to as “Okabe”).
With the method for estimating the light source according to Japanese Patent Laid-Open No. 2008-33531, it is necessary for the user to move in the entire periphery direction in order to acquire the entire periphery image, and therefor, it is not possible to easily perform the method.
Further, in the case where the method for estimating the light source by Okabe is used, the number of light source parameters is large, therefore, there has been such a problem that optimization takes much time. Furthermore, there has been such a problem that the light source parameter results in a local solution, thereby the processing becomes unstable, as is known generally in the optimization question, in the case where the difference between the CG image and the actually photographed image is minimized.